Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add runtime allocation of PF rechit fraction SoA #46135

Merged
merged 3 commits into from
Oct 3, 2024
Merged

Add runtime allocation of PF rechit fraction SoA #46135

merged 3 commits into from
Oct 3, 2024
+94 −31

Conversation

jsamudio
Copy link
Contributor

@jsamudio jsamudio commented Sep 26, 2024 •
edited
Loading

PR description:

This PR is made to address crashes of the type in #45477 and #44634. It provides runtime allocation of the PF RecHit Fraction SoA to minimize memory usage and avoid crashes when the allocation needed was not a reasonable multiple of the number of PF RecHits. This change is made in a way which does not touch the configuration, and therefore is transparent to the HLT menu.

PR validation:

PR physics performance was validated using the matrix workflow 12434.423 and checking output Legacy CPU vs GPU comparisons. The event throughput was tested on a HLT test machine with the following results:

Running 3 times over 6300 events with 8 jobs, each with 32 threads, 32 streams and 1 GPUs

Reference: 565.8 ±   0.3 ev/s

This PR: 564.7 ±   1.3 ev/s

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Not a backport.

@fwyzard @waredjeb

@cmsbuild cmsbuild added this to the CMSSW_14_2_X milestone Sep 26, 2024
@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 26, 2024 •
edited
Loading

cms-bot internal usage

@cmsbuild
Copy link
Contributor

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46135/41924

Code check has found code style and quality issues which could be resolved by applying following patch(s)

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46135/41925

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @jsamudio for master.

It involves the following packages:

  • RecoParticleFlow/PFClusterProducer (reconstruction)

@cmsbuild, @jfernan2, @mandrenguyen can you please review it and eventually sign? Thanks.
@felicepantaleo, @hatakeyamak, @lgray, @missirol, @mmarionncern, @rovere, @sameasy, @seemasharmafnal this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@jsamudio jsamudio force-pushed the dev_pfrhfAlloc_14_2_X branch from 2a00918 to 12be989 Compare September 26, 2024 14:35
@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46135/41926

@cmsbuild
Copy link
Contributor

Pull request #46135 was updated. @cmsbuild, @jfernan2, @mandrenguyen can you please check and sign again.

@jsamudio jsamudio mentioned this pull request Sep 26, 2024
Merged
@hatakeyamak
Copy link
Contributor

tagging some PF group people for information @swagata87

@jfernan2
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

+1

Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-37eb71/41773/summary.html
COMMIT: 12be989
CMSSW: CMSSW_14_2_X_2024-09-26-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46135/41773/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

@jsamudio
Rename variables and adjust allocation syntax
cdedfc6 
Adjust formatting

Rename data member and remove unnecessary include
@jsamudio jsamudio force-pushed the dev_pfrhfAlloc_14_2_X branch from 4b470e0 to cdedfc6 Compare October 2, 2024 18:38
@cmsbuild
Copy link
Contributor

cmsbuild commented Oct 2, 2024

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46135/42011

@cmsbuild
Copy link
Contributor

cmsbuild commented Oct 2, 2024

Pull request #46135 was updated. @cmsbuild, @fwyzard, @jfernan2, @makortel, @mandrenguyen can you please check and sign again.

@fwyzard
Copy link
Contributor

fwyzard commented Oct 2, 2024

enable gpu

@fwyzard
Copy link
Contributor

fwyzard commented Oct 2, 2024

please test

@fwyzard
Copy link
Contributor

fwyzard commented Oct 2, 2024

+heterogeneous

@cmsbuild
Copy link
Contributor

cmsbuild commented Oct 2, 2024

+1

Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-37eb71/41900/summary.html
COMMIT: cdedfc6
CMSSW: CMSSW_14_2_X_2024-10-02-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46135/41900/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

GPU Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 24 differences found in the comparisons
  • DQMHistoTests: Total files compared: 7
  • DQMHistoTests: Total histograms compared: 53015
  • DQMHistoTests: Total failures: 811
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 52204
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
  • Checked 24 log files, 30 edm output root files, 7 DQM output files
  • TriggerResults: found differences in 1 / 6 workflows

@mandrenguyen
Copy link
Contributor

+1
jsamudio Since heterogenous signed the comments were presumably addressed. If you have a moment, can you respond/resolve the conversations for the record

@cmsbuild
Copy link
Contributor

cmsbuild commented Oct 3, 2024

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

@cmsbuild cmsbuild merged commit 6b4f1d4 into cms-sw:master Oct 3, 2024
14 checks passed
@jsamudio
Copy link
Contributor Author

jsamudio commented Oct 3, 2024

@mandrenguyen I went through and responded/resolved the comments.

@cmsbuild cmsbuild mentioned this pull request Oct 3, 2024
Closed
@mmusich mmusich mentioned this pull request Oct 7, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@missirol missirol missirol left review comments

@mmusich mmusich mmusich left review comments

@fwyzard fwyzard fwyzard requested changes

@makortel makortel makortel left review comments

Assignees
No one assigned
Labels
code-checks-approved fully-signed heterogeneous-approved hlt-approved orp-approved pf reconstruction-approved tests-approved
Projects
None yet
Milestone
CMSSW_14_2_X
Development

Successfully merging this pull request may close these issues.
10 participants
@jsamudio @cmsbuild @hatakeyamak @jfernan2 @mmusich @swagata87 @fwyzard @mandrenguyen @makortel @missirol